Context
Greatschools is a nonprofit organization which has as an objective to provide parents, schools, and communities with high-quality information. Currently, they provide both summary ratings and sub-ratings for schools in all states in the US. Ratings are shown in a 1 to 10 scale, with only integer values.
The objective of this study is to analyze the underlying continuous variable that is used to build these ratings, both for summary ratings as well as sub-ratings in different areas.
Summary Ratings
Summary ratings are built as a weighted average of sub-ratings for each school. For example:
\[CR_{its} = \omega^{T}_{its}\cdot TestRating_{its} + \omega^{G}_{its} GrowthRating_{its} + \omega^{E}_{its} EquityRating_{its} + \omega^{C}_{its} CollegeRating_{its}\] where \(CR_{its}\) is the continuous variable used to build the summary rating for school \(i\) in time \(t\) and state \(s\), which is the weighted average of four different ratings: Test scores, Growth or Student Progress, Equity, and College Readiness. Each \(\omega\) represents a weight for each school and each rating1. Finally, the summary rating \(R_{its}\) is built from the rounding of the continuous variable:
\[\lfloor CR_{its} \rceil = R_{its}\]
Preliminary Results
Effect of ratings on Black students
Using 2017 ratings2, we estimate the regression discontinuity for the total number of Black students for the years 2018 and 2019.
| N Black students 2018 | N Black students 2019 | |
|---|---|---|
| 1-2 | 1984.362 | 2255.177 |
| (3197.054) | (3290.517) | |
| 2-3 | 6166.772** | 5877.814** |
| (2448.605) | (2456.085) | |
| 3-4 | -486.278 | 460.313 |
| (1408.997) | (1426.676) | |
| 4-5 | -53.832 | 52.342 |
| (1253.079) | (1188.699) | |
| 5-6 | -647.266 | -721.972 |
| (878.268) | (830.151) | |
| 6-7 | -2371.446*** | -2241.594*** |
| (623.801) | (613.38) | |
| 7-8 | -1805.489** | -1635.687** |
| (830.18) | (788.52) | |
| 8-9 | -516.575 | -571.766 |
| (775.344) | (761.971) | |
| 9-10 | 3045.156 | 4928.784 |
| (2401.36) | (3559.018) |
We can also plot the RDs for the entire distribution of continuous ratings, where the cutoffs are for the midpoint between each integer rating (e.g. 2.5 is the cutoff between ratings 2 and 3). Below we show the plots for both 2018 and 2019 (Note: The plots are interactive, so you can zoom in to observe the discontinuities).